Apparatus and method for transmitting media using either network efficient protocol or a loss tolerant transmission protocol

ABSTRACT

A method and apparatus for transmitting media from a node to one or more recipients over a network using either a network efficient protocol or a loss tolerant protocol. The method includes ascertaining at the node if media available for transmission is or will be consumed in real-time. When the media is or will be consumed in real-time and the condition on the network is not sufficient for the consumption of the media in real-time using the network efficient protocol, then the media is transmitted using the loss tolerant protocol. Alternatively, the media is transmitted using the network efficient protocol when the media is not or will not be consumed in real-time or the condition on the network is sufficient for the consumption of the media in real-time using the network efficient protocol. The apparatus, which may be either a communication device or server, implements the above-describe method.

This application claims the benefit of U.S. Provisional PatentApplication No. 61/323,609, filed Apr. 13, 2010, entitled “CommunicationServices Network and Client Enabled Communication Devices,” which isincorporated herein by reference for all purposes.

BACKGROUND

1. Field of the Invention

This invention relates to communications, and more particularly, to acommunication apparatus and method for transmitting media between nodesusing either a network efficient transmission protocol or a losstolerant transmission protocol, depending on (i) the condition on thenetwork and (ii) if either the transmitted media is being consumed inreal-time or in a time-shifted mode.

2. Description of Related Art

The Transmission Control Protocol or “TCP” is an example of a networkefficient protocol. TCP guarantees the delivery of transmitted databetween a sender and a recipient at the expense of speed. For thisreason, TCP is currently the most common delivery protocol used on theInternet. A feature called “flow control” is the main reason why TCP isable to guarantee the delivery of media. Flow control determines whendata needs to be re-sent and stops the flow of data until previouspackets are successfully transferred. For example, when a recipientreceives a defective packet or a packet is not received (i.e., a missingpacket), a request for retransmission of the defective and/or missingpacket is made and flow of subsequent packets is stopped until theretransmission request is satisfied. The guaranteed delivery feature ofTCP is beneficial for certain applications, such as the transfer of thecontent of web pages, files or database information. The possibilitythat the flow of data may be stopped, however, makes TCP less than idealfor delivery of time critical media, such as streaming voice or video.

The User Datagram Protocol or “UDP” is an example of a loss tolerantprotocol, commonly used on the Internet for streaming audio, video andother time-based media (i.e., media that changes over time). UDP ismainly concerned with the delivery of the most recently available media,as opposed to quality. To achieve the necessary delivery rate forstreaming audio or video, there is no form of flow control or errorcorrection with UDP. Without any mechanism for guaranteeing delivery,packets may be received out of order, defective, or lost altogether,possibly resulting in reduced quality of the media delivered to therecipient. As a result when the condition on the network are poor, mediamay be delivered at a rate sufficient for real-time consumption usingUDP when TCP would otherwise be inadequate.

Conventional communication systems are typically either real-time ortime-shifted. Consequently, a protocol like UDP, which is optimized for“real-time” delivery, is typically used for real-time systems, while aprotocol like TCP, which is optimized for reliable delivery, is used fortime-shifted systems.

SUMMARY OF THE INVENTION

This invention relates to a method and apparatus for transmitting mediafrom a node to one or more recipients over a network using either anetwork efficient protocol or a loss tolerant protocol. The methodincludes ascertaining at the node if media available for transmission isor will be consumed in real-time. When the media is or will be consumedin real-time and the condition on the network is not sufficient for theconsumption of the media in real-time using the network efficientprotocol, then the media is transmitted using the loss tolerantprotocol. Alternatively, the media is transmitted using the networkefficient protocol when the media is not or will not be consumed inreal-time or the condition on the network is sufficient for theconsumption of the media in real-time using the network efficientprotocol. The apparatus, which may be either a communication device orserver, implements the above-describe method.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention may best be understood by reference to the followingdescription taken in conjunction with the accompanying drawings, whichillustrate specific embodiments of the invention.

FIG. 1 is an architecture diagram of the communication services networkaccording to the invention.

FIG. 2 is a block diagram of a client application used in thecommunication services network of the present invention.

FIG. 3 is a media flow diagram on a client device running the clientapplication of the present invention.

FIG. 4 is a diagram of a server architecture used in the communicationsystem of the present invention.

FIG. 5 is a flow diagram for transmitting media according to the presentinvention.

FIG. 6 is a timing diagram illustrating the transmission of mediabetween sending and receiving nodes according to the present invention.

It should be noted that like reference numbers refer to like elements inthe figures.

DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

The invention will now be described in detail with reference to variousembodiments thereof as illustrated in the accompanying drawings. In thefollowing description, specific details are set forth in order toprovide a thorough understanding of the invention. It will be apparent,however, to one skilled in the art, that the invention may be practicedwithout using some of the implementation details set forth herein. Itshould also be understood that well known operations have not beendescribed in detail in order to not unnecessarily obscure the invention.

The term “media” as used herein is intended to broadly mean virtuallyany type of media, such as but not limited to, voice, video, text, stillpictures, sensor data, GPS data, or just about any other type of media,data or information.

As used herein, the term “conversation” is also broadly construed. Inone embodiment, a conversation is intended to mean a thread of messages,strung together by some common attribute, such as a subject matter ortopic, by name, by participants, by a user group, or some other definedcriteria. In another embodiment, the messages of a conversation do notnecessarily have to be tied together by some common attribute. Ratherone or more messages may be arbitrarily assembled into a conversation.Thus a conversation is intended to mean two or more messages, regardlessif they are tied together by a common attribute or not.

A. System Architecture

Referring to FIG. 1, a block diagram of the communication systemaccording to the invention is shown. The system 10 includes acommunication services network 12, a circuit switched (CS) network 14such as the Public Switched Telephone Network (PSTN), a cellular ormobile phone network, a Push to Talk (PTT) network, or a combinationthereof, and a gateway 16 coupling the two networks 12, 14. Thecommunication services network 12 includes one or more servers 18. Thecircuit switch network 14 includes, depending on the type of network orcombination of networks, one or more circuit switches 22 and possiblyone or more radio towers 24.

The communication services network 12 is IP based and layered over oneor more communication networks (not illustrated), such as the PublicSwitched Telephone Network (PSTN), a cellular network based on CDMA orGSM for example, the Internet, an intranet or private communicationnetwork, a tactical radio network, a satellite radio network, a firstresponder network, or any other communication network, or a combinationthereof. In various embodiments, the communication services network 12is either heterogeneous or homogeneous.

One or more legacy communication devices 26 are connected to the circuitswitched network 14 through either wired or wireless connection(s) 28,as is well known in the art. In various embodiments, the legacydevice(s) may include conventional land-line telephones, mobile orcellular phones, PTT radios, satellite phones or radios, desktop ormobile computers, or any combination thereof.

One or more client application 30 enabled communication devices 32 arecoupled to the communication services network 12 through an IP-basednetwork connection 34. Depending on the type of communication device 32,the connection 34 may be wired (e.g., Ethernet) or wireless (e.g.,Wi-Fi, PTT, radio, satellite, CDMA, GSM, etc.). In various embodiments,the client application 30 enabled communication devices 32 may be anytype of telephone, including both land-line, cellular or mobile phones,a PTT radio, satellite based communication device, any type of computer,including but not limited to desktop, laptop, note pad computers, or anyother type of wired or wireless communication device.

B. Client Application Architecture

The client application 30 is a messaging application that operates in atime-shifted mode, a real-time mode, and provides the ability toseamlessly transition between the two modes. With the client application30, both inbound and outgoing media is simultaneously and progressivelystored as it is either (i) received over a network connection 34 at thecommunication device 32 or (ii) created on the communication device 32and transmitted over the network connection 34. The storage of the mediaallows the participants to converse in a time-shifted mode, providing auser experience similar to conventional messaging systems (e.g., emailor voice Short Messaging Service (SMS)). The simultaneous andprogressive nature of the application 30 enables the participants toconverse “live” in a real-time mode, providing a user experience similarto a full-duplex telephone conversation.

The simultaneous and progressive storage of both transmitted media as itis being created and received media as it is being received furtherenables a host of rendering options. Such rendering options include, butare not limited to: the real-time rendering of media as the media isreceived over the network connection 34, pause, replay, play faster,play slower, jump backward, jump forward, catch up to the most recentlyreceived media, Catch up to Live (CTL), or jump to the most recentlyreceived media. As described in more detail below, the storage of mediaand certain rendering options allow the participants of a conversationto seamlessly transition the conversation from the time-shifted mode tothe real-time mode and vice versa. In addition, the client application30 is capable of supporting multiple types of media, including but notlimited to, voice, video, text, still pictures, sensor data, GPS data,or just about any other type of media, data or information.

Referring to FIG. 2, a logic block diagram of the client application 30is shown. The application 30 includes a Multiple Conversation ManagementSystem (MCMS) module 40, a Store and Stream module 42, and an interface44 provided between the two modules. In one embodiment, the MCMS module40 and the Store and Stream module 42 communicate through interface 44using an encapsulation format (e.g., JSON or XML) over a transportheader protocol (e.g., Hypertext Transfer Protocol or HTTP). Theapplication 30 functionally is similar to the client applicationdescribed in U.S. application Ser. Nos. 12/028,400 (U.S. Publication No.2009/0003558), 12/253,833 (U.S. Publication No. 2009/0168760),12/253,820 (U.S. Publication No. 20090168759) and 12/253,833 (U.S.Publication No. 2009/0168760), all of which are incorporated byreference herein for all purposes. The individual modules are brieflydescribed below. For more details, the above-listed, co-pendingapplications should be reviewed.

The MCMS module 40 includes a number of modules and services forcreating, managing and conducting multiple conversations. The MCMSmodule 40 includes a user interface module 40A for enabling the user tointerface and control the audio and video rendering and creatingfunctions on the device 32, rendering/encoding module 40B for performingrendering and encoding tasks, a contacts service 40C for managing andmaintaining information needed for creating and maintaining contactlists (e.g., telephone numbers and/or email addresses), a presencestatus service 40D for both sharing the online status of the user of thedevice 32 as well as the online status of the other users on thecommunication services network 12. The MCMS data base 40E stores andmanages the meta data for messages and conversations conducted using theapplication 30 running on a device 32 as well as contact and presencestatus information. In alternative embodiments, the MCMS database 40Emay include, but is not limited to, relational databases, file-baseddatabases, object databases, document-oriented databases, or any othertype of database and/or database management system that is capable ofstoring and retrieving data.

The Store and Stream module 42 includes a Permanent Infinite MemoryBuffer or PIMB 46 for storing, in an indexed format, the media ofreceived and sent messages. The Store and Stream module 42 also includesan encode-receive module 42A, net receive module 42B, transmit module42C and a render module 42D. The encode-receive module 42A performs thefunction of receiving, encoding, indexing and storing in a time-indexedformat in the PIMB 46 media created using the client application 30 ondevice 32. The net receive module 42B performs the function of indexingand storing in the time-indexed format in the PIMB 46 the mediacontained in messages received from other devices 32 or 26 through agateway client 16. The transmit module 42C is responsible for bothstoring in the PIMB and transmitting to recipients the media of messagescreated using the device 32. The render module 42D enables the clientapplication 30 to render on device 32 the media of messages, either inthe near real-time mode as media is received over the network 12 or inthe time-shifted mode by retrieving and rendering the media stored inthe PIMB 46.

The MCMS module 40 and the Store and Stream module 42 also eachcommunicate with various hardware components 48 provided on the device32, including, but not limited to, encoder/decoder hardware 48A, networkinterface 48B for connecting the device 32 to network connection 34, andmedia drivers 48C. The encoder/decoder hardware 48A is provided forencoding the media, such as voice, text, video or sensor data, generatedby a microphone, camera, keyboard, touch-sensitive display, GPS, sensor,etc. provided on or associated with the device 32 and decoding similarmedia before it is rendered on the device 32. The media drivers 48C areprovided for driving the media generating components, such as speakerand/or a display (not illustrated) after the media has been decoded. Thenetwork interface 48B is provided for the connecting device 32 to anetwork connection 34, either through a wireless or wired connection.Although not illustrated, the client application 30 runs or is executedby an underlying processor embedded in device 32, such as amicroprocessor or microcontroller.

The transmitted and received media stored in the PIMB 46 is persistentlystored. The term persistent storage as used herein is intended to havebroad meaning. In various embodiments, persistent storage is intended tomean the storage of media and meta data from just beyond transientstorage needed to either transmit or render media in real-time tostorage for an indefinite period of time. The term persistent storagetherefore may have different meanings, depending specificimplementations or embodiments.

In addition, “real-time” is intended to mean the consumption orrendering of time-based media (i.e., media that changes over time) asthe media is being transmitted, regardless if the media is “live” ornot. The real-time consumption of “live” media is intended to mean therendering of time-based media as the media is being created andtransmitted. The real-time consumption of non-live media is intended tomean the consumption of previously recorded time-based media that isbeing transmitted out of storage.

Referring to FIG. 3, a media flow diagram on a communication device 32running the client application 30 is shown. The diagram illustrates theflow of media for both the transmission and receipt of media, in eitherthe real-time mode or the time-shifted mode.

(i) The Receipt of Media: Media received from the communication servicesnetwork 12 is simultaneously and progressively stored in the PIMB 46 bythe net receive module 42B as the media is over the network connection34, as designated by arrow 50, regardless if the media is to be renderedin real-time or in the time-shifted mode. When in the real-time mode,the media is also simultaneously and progressively provided by the netreceive module 42B, as designated by arrow 52, to the render module 42D.In response, the render module 42D simultaneously and progressivelyrenders the media as it is received over connection 34 on amedia-rendering device (e.g., a speaker and/or display). In thetime-shifted mode, the media is retrieved by the render module 42D, asdesignated by arrow 54, from the PIMB 46 at an arbitrary time after itwas persistently stored. The retrieved media is then rendered on themedia rendering devices, such as a speaker and/or display. In thismanner, the recipient of the media may review persistently stored mediaat any time after storage in the time-shifted mode.

(ii) Transmitting Media: Media created on device 32 by a media creatingdevice (e.g. a microphone, keyboard, video and/or still camera, a sensorsuch as a thermometer or GPS, or any combination thereof) isprogressively stored in the PIMB 46 in a time-indexed format as it iscreated, as designated by arrow 58. In most situations, the media isalso provided, as designated by arrow 56, to the transmit module 42C,which simultaneously and progressively transmits the media as it iscreated. In other situations, media may be transmitted by transmitmodule 42C out of the PIMB 46 at some arbitrary time after it wascreated, as designated by arrow 60. Transmissions out of the PIMB 46typically occur when media is created when the device 32 is disconnectedfrom the network 12. When the device 32 reconnects, the media is readfrom the PIMB 46 and transmitted by the transmit module 42C.

As a clarification, the media creating devices (e.g, microphone, camera,keyboard, etc.) and media rendering devices (e.g., speaker and display)as illustrated in FIG. 4 are intended to be symbolic of such devices. Itshould be understood such devices are typically embedded incommunication devices 32, such as mobile or cellular phones, radios,mobile computers, etc. With other types of communication devices 32,such as desktop computers, the media rendering or generating devices maybe either embedded in or plug-in accessories.

C. Communication Services Network

Referring to FIG. 4, a block diagram of the communication servicesnetwork 12 is shown. As noted above, the network 12 includes one or moreservers 18. In a non-exclusive embodiment, each server 18 includes oneor more routers 20, one or more header stores 62, and one or more bodystores 64.

The routers 20 communicate with other routers 20, to header stores 62for read and/or write operations, to body stores 64 for read and/orwrite operations. Routers 20 are further responsible for updatingrouting tables and maintaining the presence status information of userson the network 12. Routers 20 also perform a number of securityfunctions, including authentication, encryption, and authorization.

In a non-exclusive embodiment, the one or more servers 18 on the network12 are highly configurable and scalable. For example, if a large numberof users subscribe to the services provided by the network 12, then alarge number of routers 20 may be needed. If a server 18 routes a highvolume of traffic, but the messages tend to be relatively short induration (e.g., contain minimal media), then the number of header stores62 may be increased relative to the number of body stores 64.Alternatively, if the traffic handled by a server tends to have largeamounts of media (i.e., the messages are long in duration), then morebody stores 64 may be needed. Further, the number of servers 18 includedon the network may be increased or reduced as needed.

On the network 12, each of the server(s) 18 subscribe to all of theheader and body data for a given user and/or group of users. As aresult, if a server 18 that holds the header and/or body information fora user becomes unavailable, a router 20 may be able to locate anotherserver 18 to obtain the data. In other embodiments, one or more users,servers or any other entity may subscribe based on the domain ofuser(s), defined sets of users, media type, codec type used to encodethe media, conversation name, conversation subject or topic, time range,or any other type of defined criteria.

D. Vox Messages and Media Payloads

Client application 30 enabled devices 32 communicate with one anotherusing individual message units, referred to herein as “Vox messages”. Bysending Vox messages back and forth over the communication servicesnetwork 12, the users of the devices 32 may communicate with oneanother, either in the real-time mode or in a time-shifted messagingmode, and with the ability to seamlessly transition between the twomodes.

There are two types of Vox messages, including (i) messages that do notcontain media and (ii) messages that do contain media. Vox messages thatdo not contain media are generally used for message meta data, such asmedia headers and descriptors, contacts information (telephone numbersor email addresses), presence status information, etc. Message meta dataincludes such attributes as a message identifier or ID, theidentification of the message originator, a recipient list, and amessage subject. The identifier information may be used for a variety ofreasons, including, but not limited to, building contact lists,associating media with messages, and/or associating messages withconversations. The Vox messages that contain media are used for thetransport of media. In one embodiment, messages containing text mediamay include both meta data and the text media, whereas messagescontaining time-based media, such as voice or video, do not contain metadata.

In one embodiment, Vox messages are layered on top of the applicationlayer of whatever transport protocol or protocols are used on theunderlying network infrastructure below the network 12. As a result, anew transport protocol for Vox messages is not needed. Rather, Voxmessages are transmitted and routed across the network 12 using currenttransport protocols running over the existing telecommunicationsinfrastructure.

The presence status information contained in Vox messages may be used toidentify the users that are currently authenticated by the system 10and/or if a given user is reviewing a message in real-time or not. Thepresence data is therefore useful in determining, in certainembodiments, how messages are delivered across the network 12. Insituations where the presence status indicates an authenticated user isreviewing a message in real-time for example, then a transport protocoloptimized for timely (i.e., real-time) delivery, such as UDP, may beused, whereas a transport protocol optimized for efficient delivery ofmessages, such as TCP, may be used when the presence status indicatesthe authenticated user is not reviewing the message in real-time.

E. Transmission Protocols and Complete Copies of Media

Referring to FIG. 5, a flow diagram 80 for transmitting media betweennodes on the communication services network 12 according to the presentinvention is shown. The sending and receiving node pair may be between acommunication device 32 and a server 18, between server 18 hops on thenetwork 12, or between any two nodes on the network. As described below,either a network efficient protocol or a loss tolerant protocol may beused, depending on (i) the condition on the network and (ii) if themedia is being consumed in real-time by the recipient.

In the initial decision 82, a transmission loop is defined at thesending node and the sending node determines if there is any mediaavailable for transmission. If not, step 82 is continually repeateduntil media becomes available for transmission. When media is available,it is next determined (decision 84) if the media is or will be consumedin real-time based on the presence information of the recipient(s). Ifthe presence information of all the recipient(s) indicates none arereviewing in real-time, then the media is transmitted using a networkefficient protocol (step 86). If one or more of the recipients isreviewing the media in real-time, then the transmitting node determinesif the condition on the network (decision 88) is sufficient fortransmitting the media at a rate sufficient to support real-timeconsumption using the network efficient protocol.

If the condition on the network is sufficient for supporting real-timecommunication using the network efficient protocol, then thetransmitting node continues transmitting using the network efficientprotocol (step 86). If the condition is not sufficient, however, thenthe transmitting node transmits the media using a loss tolerant protocol(step 90).

With media that is transmitted using the loss tolerant protocol, it isdetermined in decision 92 if the condition on the network issufficiently good enough (i.e., the rate of media loss is minimal) tosupport real-time communication. If the condition on the network is notsufficient, meaning real-time communication is not possible or practicalusing even the loss tolerant protocol, then the transmitting node stopsusing the loss tolerant protocol. Instead, the media is transmittedusing the network efficient protocol (step 86) as the condition of thenetwork permits. On the other hand if the condition on the network issufficiently good enough to support real-time communication, then themedia is transmitted using the loss tolerant protocol.

In decision 94, it is determined if the message has ended or if therecipient is no longer reviewing in the real-time mode. If neithercondition is met, then another transmission loop is defined (decision82) and the above process is repeated. When either condition is met, themedia originally transmitted using the loss tolerant protocol isretransmitted when network conditions permit using the network efficientprotocol, which guarantees the eventually delivery of a complete copy ofthe media as originally encoded. In most situations, the retransmissionoccurs when bandwidth on the network is available beyond what is neededto support real-time communication. In one embodiment, a complete copyof the media previous sent using the loss tolerant protocol isretransmitted. In an alternative embodiment, just the missing media istransmitted.

The aforementioned process is continually repeated while media isavailable for transmission. With each cycle, a transmission loop isdefined and the above-described process is repeated.

The transmission protocol as described above with respect to FIG. 5resides in layer above or on top of the underlying network efficient andloss tolerant protocols. In one embodiment, the network efficientprotocol is used as the default protocol, meaning when a transmissionbegins, the network efficient protocol is initially used and the losstolerant protocol is used only if the media is being consumed inreal-time and the condition on the network is not good enough to supportreal-time consumption. In an alternative embodiment, the loss tolerantprotocol may be the protocol that is initially used when a transmissionbegins and the network efficient protocol is used only when either (i)the media is not being consumed in real-time and/or (ii) the conditionon the network is good enough to support real-time communication usingthe network efficient protocol. The second embodiment is typically usedin situations when it is known that the transmitted media is or willlikely be consumed in real-time and/or the condition on the network istypically inadequate to support real-time consumption.

In a specific embodiment using TCP and UDP, the transmission protocol asdescribed above with respect to FIG. 5 monitors the rate at which theTCP protocol transmits media. So long as the transmit buffer used by theTCP protocol does not back-up beyond a predetermined threshold oroverflow, TCP is used for transmissions, regardless if the media isbeing consumed in real-time or not. If the transmission buffer backs-upbeyond the predetermined threshold level or overflows, indicating thatthe condition on the network is inadequate to support the transmissionof media at a rate sufficient to support real-time consumption, then aswitch occurs and UDP is used for media that is being consumed inreal-time. The transmitting node will continue using UDP until (i) thetransmitted media is no longer being consumed in real-time, (ii) thecondition on the network has improved to the point where TCP may be usedfor the transmission and consumption of the media in real-time; and/or(iii) conditions on the network are so poor, it is not practical for themedia to be consumed in real-time even using UDP.

The transmission protocol as described above with respect to FIG. 5resides at both communication devices 32 and at servers 18 on thenetwork 12. In a non-exclusive embodiment, the transmission protocol isembedded in the client application 30. In an alternative embodiment, thetransmission protocol may be separate from but cooperate with theapplication 30. Regardless of how implemented, the transmission protocolas described herein allows both client 30 communication devices 32 andserver(s) 18 to communicate with one another using either a losstolerant or network efficient protocol, depending on if the media beingtransmitted is being consumed in real-time and the condition of thenetwork.

Referring to FIG. 6, a timing diagram illustrating the transmission ofmedia between sending and receiving nodes is illustrated. Initially attime T1, client 32A begins transmitting the media of a message (step 82)to recipient client 32B using the network efficient protocol (e.g.,TCP). At time T2, the presence information of the recipient 32Bindicates if the media of the message is being reviewed in real-time ornot. Based on the presence information (decision 84), the sending nodeevaluates if the media is being consumed in real-time or not. In thistiming diagram example, it is assumed that the media is in fact beingconsumed in real-time and the network is not good enough (decision 88)to support real-time communication using the network efficient protocol.As a result at time T3, the media of the message is transmitted usingthe loss tolerant protocol (step 90). Eventually the message ends and/orthe recipient transitions from the real-time to the time-shifted mode.When either event occurs, as designated by the time T4, the transmittingclient 32A retransmits, according to different embodiments, either justthe missing media or the media previously sent using the loss tolerantprotocol. In either case, the network efficient protocol is used,guaranteeing the delivery of a complete copy of the media of the messageto the recipient 32B.

The timing diagram is intended to illustrate the asynchronous nature ofthe transmission flow diagram illustrated in FIG. 5. Many of the stepsand/or decisions outlined in the flow diagram of FIG. 5 occur inparallel to the extent possible. Many of the events as described overlapwith one another to some extent and do not necessarily occur in“lock-step”, meaning it is not necessary for one step to start onlyafter the previous step is complete, as is typical with most synchronous“clock-driven” communication systems. Rather, each step is started assoon as the conditions necessary to perform that step are completed.This is particularly true with the transmission of media for example.Conventional communication systems typically transmit media using only asingle transmission protocol. With the protocol as described herein,however, the transmission of media may be switched “on the fly” duringthe course of a message between either the network efficient or losstolerant protocols, depending on the presence status of the recipient(s)and/or condition on the network.

The transmission of media between communication devices 32, as describedabove, occurs through one or more servers 18 on the network 12. In analternative pier-to-pier embodiment however, it would be possible fortwo communication devices 32 to communicate directly with one another.With this embodiment, the flow and timing diagrams of FIGS. 5 and 6would still be applicable, except the communication flow would occurdirectly over the network 12 between a sending communication device 32and one or more recipient communication devices 32.

It also should be noted that for the sake of simplicity, thetransmission of messages as described above has been “one-way”. Itshould be understood that the transmissions of media, using either thenetwork efficient and/or the loss tolerant protocol, is oftenbi-directional or “two-way”. With two-way communication in the real-timemode, the user experience is similar to a conventional full-duplexconversation. In addition, bi-directional communication can also takeplace between multiple parties (i.e, more than two), similar to amulti-party conference call.

By using either the network efficient or loss tolerant protocol,depending on if media is being consumed in real-time and/or on thecondition of the network, transmissions are optimized for real-time whenneeded or for efficient delivery when real-time delivery is not criticalor network conditions are sufficiently good to support real-timecommunication using the network efficient protocol. On the other handwhen the media is being reviewed in real-time and network conditions arepoor, the loss tolerant protocol is typically used to extend or enhancethe ability to conduct real-time communication. Since any missing ormedia transmitted using the loss tolerant protocol is eventuallyretransmitted using a network efficient protocol that guarantees thedelivery, the recipient eventually receives a complete copy (i.e. a fullbit rate representation as the media was originally encoded). The fullbit rate representation of the media typically replaces any missing,defective or out of order representations of media previously receivedand stored in the PIMB 46 of the receiving device 32. In this manner,the recipient may review the complete, full quality, version of themedia in the time-shifted mode at a later arbitrary time.

In one embodiment, TCP is used as the network efficient protocol, whileUDP is used as the loss tolerant protocol. In other embodiments, TCP isused as the network efficient protocol, while the CooperativeTransmission Protocol (CTP) described in U.S. application Ser. No.12/192,890, incorporated by reference herein, is used as the losstolerant protocol. In other embodiments, any network efficient or losstolerant protocol may be used.

Although many of the components and processes are described above in thesingular for convenience, it will be appreciated by one of skill in theart that multiple components and repeated processes can also be used topractice the techniques of the system and method described herein.Further, while the invention has been particularly shown and describedwith reference to specific embodiments thereof, it will be understood bythose skilled in the art that changes in the form and details of thedisclosed embodiments may be made without departing from the spirit orscope of the invention. For example, embodiments of the invention may beemployed with a variety of components and should not be restricted tothe ones mentioned above. It is therefore intended that the invention beinterpreted to include all variations and equivalents that fall withinthe true spirit and scope of the invention.

1.-20. (canceled)
 21. A method for transmitting media over a networkcomprising: transmitting the media over the network using a networkefficient protocol when either (i) the media is not being consumed inreal-time or (ii) the media is being consumed in real-time and theavailable bandwidth on the network is good enough to support atransmission rate suitable for the real-time consumption of the mediausing the network efficient protocol; or transmitting the media over thenetwork using a loss tolerant protocol when (iii) the media is beingconsumed in the real-time mode and (iv) the available bandwidth on thenetwork is not good enough to support a transmission rate suitable forthe real-time consumption of the media using the network efficientprotocol.
 22. The method of claim 21, further comprising: ascertainingthe available bandwidth on the network; and transmitting the media usingeither the network efficient protocol or the loss tolerant protocoldepending on the ascertained available bandwidth on the network.
 23. Themethod of claim 21, further comprising: identifying the mediatransmitted using the loss tolerant protocol; and re-transmitting atleast a portion of the identified media using the network efficientprotocol when the bandwidth on the network is available.
 24. The methodof claim 23, wherein re-transmitting at least a portion of theidentified media comprises one of the following: (i) re-transmitting theidentified media in full; or (ii) re-transmitting only the media lostwhen transmitting using the loss tolerant protocol.
 25. The method ofclaim 21, wherein the loss tolerant protocol is UDP.
 26. The method ofclaim 21, wherein the network efficient protocol is TCP.
 27. The methodof claim 21, wherein the network efficient protocol guarantees thedelivery of the media over the network.
 28. The method of claim 21,further comprising: ascertaining the presence status of one or morerecipients of the media; and ascertaining if the transmitted media is tobe consumed in real-time based on the presence status of the one or morerecipients respectively.
 29. The method of claim 21, wherein theconsumption of the media in real-time further comprises one of thefollowing: (i) the rendering of the media “live” as the media is createdand transmitted; or (ii) the rendering of previously created and storedmedia streamed from storage.
 30. The method of claim 21, furthercomprising: defining the network efficient protocol as a defaultprotocol when initially transmitting the media; and switching thetransmission of the media from the network efficient protocol to theloss tolerant protocol when: the media is or will be consumed inreal-time; and the available bandwidth on the network is not good enoughto support the transmission of the media at a rate suitable forconsumption in real-time using the network efficient protocol.
 31. Themethod of claim 21, further comprising: defining the loss tolerantprotocol as a default protocol when initially transmitting the media;and switching the transmission of the media from the loss tolerantprotocol to the network efficient protocol when either: (i) the media isnot being consumed in real-time; or (ii) the media is being consumed inreal-time and the available bandwidth on the network is good enough tosupport the transmission of the media at a rate suitable for consumptionin real-time using the network efficient protocol.
 32. The method ofclaim 21, further comprising ascertaining the available bandwidth on thenetwork by determining when a transmission buffer used for buffering themedia for transmission exceeds a predetermined threshold or overflowsduring the transmission of the media.
 33. The method of claim 21,wherein the transmitted media consists of voice, video, still pictures,text, GPS or position data, sensor data, or any combination thereof. 34.A communication device, comprising: a transmission element, thetransmission element configured to: transmit media over a network usinga network efficient protocol when either (i) the media is not beingconsumed in real-time or (ii) the media is being consumed in real-timeand the available bandwidth on the network is good enough to support atransmission rate suitable for the real-time consumption of the mediausing the network efficient protocol; or transmit the media over thenetwork using a loss tolerant protocol when the media is being consumedin the real-time mode and the available bandwidth on the network is notgood enough to support a transmission rate suitable for the real-timeconsumption of the media using the network efficient protocol.
 35. Thedevice of claim 34, wherein the transmission element is furtherconfigured to: ascertain the available bandwidth on the network; andtransmit the media using either the network efficient protocol or theloss tolerant protocol depending on the ascertained available bandwidthon the network.
 36. The device of claim 34, wherein the transmissionelement is further configured to: identify the media transmitted usingthe loss tolerant protocol; and re-transmit at least a portion of theidentified media using the network efficient protocol when the bandwidthon the network is available.
 37. The device of claim 36, wherein thetransmission element, when re-transmitting the at least a portion of theidentified media, is further configured to: (i) re-transmit theidentified media in full; or (ii) re-transmit only the media lost whentransmitting using the loss tolerant protocol.
 38. The device of claim34, wherein the loss tolerant protocol is UDP.
 39. The device of claim34, wherein the network efficient protocol is TCP.
 40. The device ofclaim 34, wherein the network efficient protocol guarantees the deliveryof the media over the network.
 41. The device of claim 34, wherein thetransmission element is further configured to: ascertain the presencestatus of one or more recipients of the media; and ascertain if thetransmitted media is to be consumed in real-time based on the presencestatus of the one or more recipients respectively.
 42. The device ofclaim 34, wherein the consumption of the media in real-time furthercomprises one of the following: (i) the rendering of the media “live” asthe media is created and transmitted; or (ii) the rendering ofpreviously created and stored media streamed from storage.
 43. Thedevice of claim 34, wherein the transmission element is furtherconfigured to: define the network efficient protocol as a defaultprotocol when initially transmitting the media; and switch thetransmission of the media from the network efficient protocol to theloss tolerant protocol when: the media is or will be consumed inreal-time; and the available bandwidth on the network is not good enoughto support the transmission of the media at a rate suitable forconsumption in real-time using the network efficient protocol.
 44. Thedevice of claim 34, wherein the transmission element is furtherconfigured to: define the loss tolerant protocol as a default protocolwhen initially transmitting the media; and switch the transmission ofthe media from the loss tolerant protocol to the network efficientprotocol when either: (i) the media is not being consumed in real-time;or (ii) the media is being consumed in real-time and the availablebandwidth on the network is good enough to support the transmission ofthe media at a rate suitable for consumption in real-time using thenetwork efficient protocol.
 45. The device of claim 34, wherein thetransmission element is further configured to ascertain the availablebandwidth on the network by determining when a transmission buffer usedfor buffering the media for transmission exceeds a predeterminedthreshold or overflows during the transmission of the media.
 46. Thedevice of claim 34, wherein the transmitted media consists of voice,video, still pictures, text, GPS or position data, sensor data, or anycombination thereof.
 47. The device of claim 34, wherein the devicecomprises one of the following: a cellular phone, a mobile phone, aland-line phone, a PTT radio, a satellite radio, a desktop computer, amobile computer, a wired communication device, a wireless communicationdevice or a server.
 48. A communication application embedded in anon-transient computer readable medium and intended to run on acommunication device, the application comprising: a transmission module,the transmission module configured to: transmit media over a networkusing a network efficient protocol when either (i) the media is notbeing consumed in real-time or (ii) the media is being consumed inreal-time and the available bandwidth on the network is good enough tosupport a transmission rate suitable for the real-time consumption ofthe media using the network efficient protocol; or transmit the mediaover the network using a loss tolerant protocol when (iii) the media isbeing consumed in the real-time mode and (iv) the available bandwidth onthe network is not good enough to support a transmission rate suitablefor the real-time consumption of the media using the network efficientprotocol.
 49. The application of claim 48, wherein the transmissionmodule is further configured to: ascertain the available bandwidth onthe network; and transmit the media using either the network efficientprotocol or the loss tolerant protocol depending on the ascertainedavailable bandwidth on the network.
 50. The application of claim 48,wherein the transmission module is further configured to: identify themedia transmitted using the loss tolerant protocol; and re-transmit atleast a portion of the identified media using the network efficientprotocol when the bandwidth on the network is available.
 51. Theapplication of claim 50, wherein the transmission module, whenre-transmitting the at least a portion of the identified media, isfurther configured to: (i) re-transmit the identified media in full; or(ii) re-transmit only the media lost when transmitting using the losstolerant protocol.
 52. The application of claim 48, wherein the losstolerant protocol is UDP.
 53. The application of claim 48, wherein thenetwork efficient protocol is TCP.
 54. The application of claim 48,wherein the network efficient protocol guarantees the delivery of themedia over the network.
 55. The application of claim 48, wherein thetransmission module is further configured to: ascertain the presencestatus of one or more recipients of the media; and ascertain if thetransmitted media is to be consumed in real-time based on the presencestatus of the one or more recipients respectively.
 56. The applicationof claim 48, wherein the consumption of the media in real-time furthercomprises one of the following: (i) the rendering of the media “live” asthe media is created and transmitted; or (ii) the rendering ofpreviously created and stored media streamed from storage.
 57. Theapplication of claim 48, wherein the transmission module is furtherconfigured to: define the network efficient protocol as a defaultprotocol when initially transmitting the media; and switch thetransmission of the media from the network efficient protocol to theloss tolerant protocol when: the media is or will be consumed inreal-time; and the available bandwidth on the network is not good enoughto support the transmission of the media at a rate suitable forconsumption in real-time using the network efficient protocol.
 58. Theapplication of claim 48, wherein the transmission module is furtherconfigured to: define the loss tolerant protocol as a default protocolwhen initially transmitting the media; and switch the transmission ofthe media from the loss tolerant protocol to the network efficientprotocol when either: (i) the media is not being consumed in real-time;or (ii) the media is being consumed in real-time and the availablebandwidth on the network is good enough to support the transmission ofthe media at a rate suitable for consumption in real-time using thenetwork efficient protocol.
 59. The application of claim 48, wherein thetransmission module is further configured to ascertain the availablebandwidth on the network by determining when a transmission buffer usedfor buffering the media for transmission exceeds a predeterminedthreshold or overflows during the transmission of the media.
 60. Theapplication of claim 48, wherein the transmitted media consists ofvoice, video, still pictures, text, GPS or position data, sensor data,or any combination thereof.